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ABSTRACT 

Presented by the Technical Assistance Development 
system are a planning model for the specification of problems and an 
evaluation strategy for information collection to implement decision 
making in the First Chance network of programs for preschool 
handicapped children, The first chapter offers a planning model in 
terms of measurable goals, administrative objectives, and outcome 
objectives for the target populations of children, parents, anJ 
decision makers. The second chapter considers evaluation to consist 
of daily process evaluation and longer term product evaluation based 
on standardized and nonstandardized measures of administrative and 
outcome objectives. Major types of data requests by the Bureau of 
Education for the. Handicapped are briefly considered in the third 
chapter. The final chapter on educational trends suggests that the 
emphasis on planning and evaluation has arisen out of research such 
as the J,S, Coleman Report as well as community doubts about the 
educational non-system's ability to improve. Planning problems such 
as funding patterns are noted. Also given are a glossary of planning 
terms and examples of goals and objectives, See Ec 050 080 for volume 
one, (DB) 
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Firs/ Chance far Children h a scrips of monographs published for the First Chance Network, The subject 
matter is drawn from the knowledge, skills, and techniques of the people that work within the First Chance 
Network and h collected and published by the Technical Assistance Development System. 



In I 963 the enactment of the Handicapped Children's Early Education Act authorized the establishment and 
operation of mode! early education projects. Collectively these piojects are referred to as the First Chance 
Network, The responsibility for administering this new program was accepted by the Bureau of Education for 
the Handicapped, Office of Education, The program is designed to develop and demonstrate effective 
approaches in assisting handicapped children during their early years and is structured so that other 
ties can replicate, or adopt, exemplary program components to meet their own needs. 
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The Bureau of Education for the Handicapped (B.E.H.) has as its overall goal the equalization of educational 
opportunity, for handicapped children by providing the leadership and resources needed to help the handi- 
rnVr capped achieve their fullest potential and participate constructively in society to their maximum abilities The 

long-range objective of the Handicapped Children's Early Education Program is to stimulate services to all 
estimated 1,000,000 preschool-aged handicapped children by the end of this decade, 



Technical Assistance Development System (TADS) was established in Chapel Hit!, N.C. by B E H to serve •, 
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conferring w.th md.v,dual centers a „d staff,. Most often, services arc offered fo First Chance projects in the 
areas of p.ogram planning and evaluation, intervention programs, community program development, and 
mcahi an'j information, r 

Thi;. monograph is distributed pursuant to a grant from the Office of Education, U.S. Department of Health 
R.ucat.on, and Welfare Grantees undertaking such projects under government sponsorship arc encouraged to 
-press freely the.r judgment in prof^bnal and technical matter, Points of view of opinions do mu 
therefore, necessardy represent official Office of Education position or policy 
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INTRODUCTION" 



Having found that the planning and evaluation of programs for preschool handicapped children was a major 
concern in the First Chance Network, the Technical Assistance Development System (TADS) staff sought to 
develop a systematic informational model for program planning and evaluation which could be utilized by 
First Chance Projects. It was obvious from the first that this model should be designed around the specific 
needs and requirements of the projects. Keeping this in mind, the TADS staff made sure that the process of 
developing materials included; 1) a search for what was being done: 2) a discussion both within TADS and 
throughout the network of what might be done; 3} the creation of a planning model with structures and 
examples; and, 4) several field tjsts of the created materials. 

Initially, TADS staff members examined pioposals and progress reports prepared for the U S Office of 
bducat.on s Bureau of Education for the Handicapped (B.E.H.) by all First Chance Centers so that the staff 
members could collect information about models and norms that were being applied. Special attention wis 
given m this examination to the content and structure of the goals and objectives. From this process it was 
determined that there was a definite lack of consistency in the statements of project plans, 

In '-a. V January, 1971 Dr, James Gallagher presented a general plan and the rationale for it to B E H 
winch called for TADS to sponsor a series of small modular meetings for project pcrconnel who requested aid 
in the areas or planning and evaluation, 



TADS PLAN FOR TECHNIC/ L ASSISTANCE DELIVERY 

The actual modilc meeting had one objective; to provide a general orientation for the center directors on 

ERICl T emS Tm- P 1 a " d evaluation that we agree upon, This orientation would 

L^lude a careful lutemng to the center directors' particular p,ob!«ns and an attempt during the two days to 



provide some initial help in the structuring of their 
objectives 

Once materials to be used in these modular 
meetings were compiled, a planning structure was 
developed and an objective matrix, based upon 
First Chance requirements, was created. Finally, 
special efforts were made to provide illustrative 
examples of most of the ideas and concepts to be 
discussed in the module meetings. 

After the first series of planning and evalua- 
tion module meetings had been held, the TADS 
staff realized that future meetings could be organ- 
ized in such a way as to provide answers to several 
frequently asked questions about planning and 
evaluation, Accordingly, materials were reorgan- 
ized, reviled, and coordinated with those ques= 
tions. 

This monograph contains the information 
presented at those meetings, The purpose of this 
presentation is to offer a planning model for the 
specification of problems and an evaluation 
strategy far the collecting of ill format ion for 
decision making. It describes one method of 
response to inquiries ibout program planning and 
evaluation. In no way is it intended as a fin J or an 
only model but it is an initial attempt to deal with 
these problems. In fact, it is hoped that the mater- 
ials will elicit constructive criticism so that further 
refinement might occur, 

This monograph is divided into four chapters 
in which the following topics are diseussedi plan- 
ning, evaluation, data, and some current trends. 
The first chapter presents one method of prob- 
lem-solving with a special emphasis on planning 
educational programs which can be subjected to 
evaluation. In the second chapter, attention is 
given to several techniques of evaluation while the 
third chapter briefly discusses the data requests 
made by the Bureau of Education for the Handi- 
capped, Finally, chapter four provides a discussion 



or educational trends as well as a review of some 
potential problems with planning and evaluation, 

J J. a 

K. C. S. 

a, e. h. 

Chapel Hill, North Carolina 
Winter 1973 
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CHAPT1IE I 



■ • ■ the method, or technique used for planning requires 
thai the planner determine the various problems, the ele- 
ments cf each problem, and how each element effects the 
other components of the situation, 

. « . to gain some idea of effectiveness one must state spe- 
cific intentions which include measurable indices and time 
limits. This can be accomplished by stating objectives, 
. , . assumption is then made that if a positive evaluation of 
the specific objective results, then the project is in the 
process of meeting its goal 




WHAT IS PLANNING AND HOW IS IT DONE? 

Planning, as defined by TADS, is a conceptual scheme for a systemmatie, problem-solving attack. Moreover 
the method, or technique, used for planning requires that the planner determine the various problems, the 
elements of each problem, and how each element effects the other components of the situation. 

To aid planners in this process, a procedural model for planning was designed, a matrix of subject areas 
was created, and examples of how one might„usc this information were developed. Special attention was given 
to the development of a precise language for speaking about planning. In this way, by specifying one model 
and one language, a consistent reference-point will exist for discussing data and programs. This model pre- 
sented in Figure 1 (page 2), contains some elements of planning that a project director or planner can use 
while developing programs for a target group such as handicapped children, parents, or decision-makers, 

The elements of this model arc all inter related. Needs alert m to potentials for change and generate 
goals. Goals require specified objectives which can only be met or realized within the boundaries of resources 
balanced by constraints, Strategies for reaching objectives, selected from alternative approaches lead to a 
choice of action an implementation activity, an evaluation of the success of the strategy, and feedback. This 
feedback ot evaluation data helps to adjust goals, improve resources, sharpen objectives or reduce constraints 

In our day-to-day lives most of us do not plan this systematically. For example, someone who has been 
trained in deaf education and who has been working with older children may decide to work with younger 
children because he has become impressed with the value of early education. In this example, this Individual 
enters the planning process having already determined the "alternative strategy", "selection of criteria" and 

PR IP" . ,° th ° modeL The P r °blen>solving approach in this case -would be to define the parameters of 

L£sJV the other seven elements in the planning model, 



FIGURE 1 

PROGRAM PLANNING AND EVALUATION MODEL 
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The following examples which describes how 
a project director might evaluate his project by 
using the model, can serve to demonstrate what 
the planning variables arc a. id how they can be 
utilized. While the example focuses on evaluation, 
one can analyze problems such as how to select 
children for programs, how to staff, how to design 
a curriculum, or how to find additional funds by 
using the model. 



CONSTRAINTS: Budget of $2,000,00 for evalua- 
tion, 

Teachers are reluctant to partici- 
pate in an evaluation. 

No one on the staff is familiar 
with data analysis involving sta= 
tistics, 



RESOURCES: 



EXAMPLE OF MODEL 



NEED: 



GOAL: 



OBJECTIVES: 
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An evaluation plan must be 
developed and carried out. 

The plan must meet B.E.R's 
requirements and must provide 
information to tilt public about 
the effectiveness of the program- 
To complete the evaluation plan 
by September 1 

To begin collecting data by 
September 15 

To complete data collection by 
April 30 

To complete the data analysis 
and to file a final report with 
B.E.H, by June 1 

To prepare a brief report for 
public dissemination that will 
outline the successes of the 
project 



ALTERNATIVE 
STRATEGIES^ 



SELECTION 
OF 

CRITERIA! 



Budget of $2,000.00 for evahia= 
tion 

A firm can provide consultative 
assistance in evaluation, 

A college in town has graduate 
students who could help with 
the data analysis, 

An evaluation consultant will 
help develop the plan. 

The project director hires a con- 
sulting firm to develop and carry 
out evaluation plan. 

The director seeks an additional 
staff person who would devote 
one-fourth of his time to evalua- 
tion. 

The director attends workshops 
on evaluation and develops plan, 

The director uses the staff in 
conjunction with the consultant, 

The consulting firm charges 
$1,700,00 to collect and analyze 
data, 



CHOICE; 



The director wants to allocate at 
least 5500,00 for printing and 
dissemination of data, but could 
gut by with $300.00. 

The consultant says that he will 
set up the plan (objectives and 
evaluation methods) and provide 
graduate students to analyze 
data for 31,000,00 

Teachers know and trust the 
consultant. 

Graduate students will be quali- 
fied testers and will not interfere 
with classroom activities. 

With the extra $1,000,00, the 
director can hire a consultant to 
help decide how best to design 
and distribute data, 



The director hires the consultant 
and graduate students for 
$1^000,00 and closely supervises 
their activities, 



Report sent in to B.E.H. by 
June 30 

Two-page data sheets ready 
for dissemination by June 30 

The director seeks information 
from B.E.H. about the quality of 
the report to them. 

The director hires an independ- 
ent consultant from next yvur*s 
budget to study the impact of 
public dissemination of data, 

FEEDBACK: A budget of $2,000.00 for evalu- 
ation is probably too small. 

The director needs to find out 
how other project directors are 
handling this problem. 

The director needs to hire a per- 
son (or next year who could 
serve as an administrative assist- 
ant and as the director of evalua- 
tion for the project, 
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EVALUATION: The director checks to see if the 
following events occur as 
planned? 

Evaluation plan created by 
September 1 

Collecting data begun by Sep- 
tember 15 

Data collected by April 30 



hi general, then, the model describes a series 
of structures, indicates that those dimensions are 
inter-related, and allows the planner to "fill-in- 
thc-blanks" with information that (Its his needs, 
The model is conceived of as one way a project 
director or planner might begin to conceptualize 
what has to be defined and accomplished. It is not 
intended to be an ideal of how one plans nor as an 
example of a theoretical hierarchy of planning. In 
fact, starting to plan is probably a very random 
process, and the model may present planning ele- 
ments in an artificial order. 
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WHAT ELEMENTS OF THE MODEL ARE 
MOST ESSENTIAL TO THE PLANNING 
AND THE SUBSEQUENT EVALUATION 
PROCESS? 

In order to use the planning nnd evaluation 
model, one should define the terms it contains. 
Definitions of each are provided in the Glossary of 
Terms (Appendix A), but special emphasis needs 
to be placed on defininfr goals, objectives; and eval- 
uation. 

Goal statements reveal the project's long- 
range intentions, for example, what is assumed will 
happen to its target population as a result of a 
certain programmed treatment? Specifically, one 
goal of a project which serves two-year-old re- 
tarded children may be to improve their overall 
cognitive development so that they are able to 
attend nnrmul classrooms during public school 
years, Obviously, this statement is filled with 
many assumptions end value judgements, but it 
gives the direction or purpose of the project's prt> 
grams, However, this type statement does not lend 
itself to evaluation of treatment effectiveness, 

To gain some idea of effectiveness, one must 
state more specific intentions which include inca* 
ureable indices and time limits. This can be 
accomplished by stating objectives which describe 
what will be done by the project during its opera- 
tions and which are related to a specified goal, The 
assumption is then made that if a positive evalua- 
tiOii of the specific objective results, then the 
project is in the process of meeting its g 0 a| 4 

Objectives described in this monograph are 
nut the type generally used for dally operations 
but they are, somewhat like goals, statements 
which encompass a broad timeframe from four 
months to one year, Since delineating tasks and 
reporting data are the major reasons for having 
these objectives, they are divided along functional 
lines which have been labeled administrative and 



outcome objectives. 

Administrative objectives reveal the manage- 
ment strategy used by the project. They indicate 
what has been done or will be done In project 
operations. For example, during the months of 
January, February and March a project might 
establish a weekly parent discussion group, hire an 
early childhood consultant, and operate three 
classes for physically handicapped children. 

Outcome objectives reveal expected changes 
in behavior or attitude such as children's gross 
motor abilities and expressive language. The effec* 
tiveness of these objectives must be described in 
evaluative, or data-based terms. 

With parents as targets. Chart 1 (page 7 ) 
presents an example of a goal, an outcome objec- 
tive, and an administrative objective with sample 
evaluation results. Since outcome objectives tend 
to be the most difficult to develop, Appendix H 
offers some additional examples, In it, outcome 
objectives are parallelled with goals, treatment 
strategies, and evaluation methods. 

Finally, the project director will need to 
make a somewhat arbitrary decision about how 
many objectives are needed to describe procedures 
and how much data should be collected in the 
evaluation of those objectives, Administrative type 
objectives are relatively easy to specify and eval- 
uate; thus, the number does not seem overly 
important, However, it is recommended that a 
project focus on one or two goals with only two or 
three outcome objectives associated with each. 
The reason for this is that the process of collect- 
ing, analyzing and describing data results can 
become overwhelming, Unless specially funded for 
data-processing, the project probably will not want 
to allocate an extensive amount of its total re- 
sources a iid time to this problem, 

Appendix C offers additional material which 
offers several other examples of the distinction 
made between goals and objectives, Also included 



Q 

arc some sample structures which might be useful 
when writing goals, administrative objectives and 
outcome objectives. 



FOR WHAT TARGET POPULATIONS 
SHOUI D PROJECTS PLAN PROGRAMS? 

Although the First Chance Network serves a 
heterogeneous population of preschool, handi- 
capped children and their parents, the projects 
have in common three major target audiences for 
which they need to develop programs i children, 
parents, and decision-makers, 

Delivery of services to children and to their 
parents are usually the first problems considered 
by a project, Having developed programs in these 
two areas, the project begins to select those in 
decision-making positions to aid in the further 
development and replication of programs for pre- 
school handicapped children, Chart 2 ( page 9 ) 
shows the relationship of the§e target groups to 
goals and objectives, Chart 3 (page 1 1) is an exam- 
ple of how this matrix might look when corr> 
pleted, 



WHY IS A PRECISE STATEMENT OF A 
PROJECT'S PLANS IMPORTANT? 

There are many reasons why specifying plans 
is useful, but three seem to stand out, They are; 

GUIDELINES, Presumably, well-stated plans 
should be an asset to a project's staff, Knowing the 
purpose and direction of an organization should 
reduce anxiety and facilitate decision-making and 
this is congruent with the goals of the project. 

INFORMATION, Well-stated plans can be 



used to communicate Co others what results a 
project expects from its activities, Targets for re- 
ceiving such information might include the fund= 
ing agency, decision-makers for replication or par= 
ems whose children participate in the project's 
progra irL 

EVALUATION, Planning is the core of the 
concept of evaluation presented in this mono- 
graph. The belief is that evaluation should be re- 
lated to statements of intent. Chapter II presents 
this idea in a detailed analysis, 



CHART 1 

EXAMPLE OF THE RELATIONSHIP OF GOALS, OBJECTIVES AND EVALUATION 



STATEMENT 



■GOAL 



♦OUTCOME 
OBJECTIVE 



'ADMINISTRATIVE 
OBJECTIVE TO 
CARRY OUT 
STRATEGY 
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EXAMPLE 



The project^ parent program 
will increase flic parent's 
knowledge, understanding, 
care and training of their ex= 
ceptional child so that the 
child is able to enroll in a nor= 
ma I classroom by age seven 



To^ereaie by 40% over base- 
line' the positive attitude of 
the parents regarding the ac- 
ceptance of the child's handi= 
capping conditions by the 
end of his first year in the 
program 



To have at least one parent 
from each family attend 75% 
of the weekly meetings dur- 
ing the school year in which 
the major topic of discussion 
will be problems dealing with 
children 



EVALUATION RESULTS 



Unkn 



own a 



t th 



is time 



As a group, parents' positive attitudes 
toward the acceptance of the child's 
condition improved by 50% as meas- 
ured by a center-made criterion ref- 
erenced tests used in a pre=po S t man- 
ner, When asked directly by staff at 
the end of the year if they felt they 
were better able to cope with the 
child, and associated problems, 95% of 
the parents said they were 



Records kept by staff indicated that 
one parent of 95% of the families at- 
tended 82% of all meetings, A content 
analysis of the anecdotal records of 
the meetings indicated that problems 
with children was the topic of discus- 
sion in 90% of the meetings 



'Natei Orher objectives related to goal are needed in actual program pnctfcc 



CHART 2 

PROGRAM AREAS FOR FIRST CHANCE PROJECT 



PLANNING 






OBJECTIVES MATRIX 


CHILDREN 


PARENTS 



GOALS 



DECISIONMAKERS 



ADMINISTRATIVE 
OBJECTIVES 



OUTCOME 
OBJECTIVES 



CHART 3 
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EXAMPLE OF COMPLETED MATRIX 





CHILDREN 


PARENTS 


DECISION MAKERS 


GOALS 


By June 1, 1976, the 
project will increase the 
developmental level of 
n o n - la nguage excep- 
tional children in five 
areas' person a Lsoeial 
skills, gross motor 
skills, fine motor skills, 
cognitive skills and lan- 
guage skills so that the 
ehildren avoid institu- 
tionalization, 


By June 1, 1976, the 
project's parent pro- 
gram will increase the 
parent's knowledge, un- 
derstanding, care, and 
teaching of their excep- 
tional children so that 
they ea n effectively 
deal with the child in 
the home 


The project will solieii 
replication of its pro- 
grams through consulta- 
tions with prospective 
replicators, and th rough 
seminars, letters, dis- 
semination and demon- 
stration activities dur- 
ing the life of the proj- 
ect 


ADMINISTRATIVE 
OBJECTIVES 


To identify, assess and 
include by November 1, 
1972, those children in 
the child-training pro- 
gram who meet the erU 
teria of age— from birth 
to five years— and ex= 
eeptionality 

To develop by Septenv 
bur I, 1972, sets of les= 
son plans to be includ- 
ed in training manuals 
for the training of ex- 
ceptional children 


To select by November 
1 , 1 973, families for 
participation in a lan- 
guage clinic 

To develop by Novem- 
ber 1, 1973, sets of les- 
son plans to be includ- 
ed in training manuals 
for the training of par= 
Gilts in the teaching of 
their exceptional child 


I f 1 n -iti K i f |i|u„ 1 

io pian oy June I, 
1973, methods of mak- 
ing the project visible 
to the general public 
and professional com- 
munity 

To plan by June |, 
1973, the composition, 
intended viewing audi- 
ence, distribution, ob- 
jectives and purposes of 
all project materials to 
be distributed, such asi 
broch ures, television 
tapes, and slides 


OUTCOME 
OBJECTIVES 


To increase by June 1, 
1973, the receptive and 
expressive language of 
all project ehildren by a 
s ign i flea n t amount as 
measured by the ITPA, 
and ecu ter- prepared, 
criterion reference test 


To increase by June 1, 
1 973, parents' knowl- 
edge of child rearing 
practices so that 95% of 
parents are able to re- 
spond to teacher-made 
tests on child rearing 
with 80%, accuracy 


To increase the com= 
munity support of pre^ 
school education by 
disseminating informa- 
tion on the project's 
3h ilosophy, success 
with young children 
and its potential worth 
to the total .community 
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. • . process evaluation provides information for daily 
decision-makers; pmdttct evaluation is a summary of infor- 
mation taken avet a longer time frame such as o fie month, 
six months, or one year, 

, i i objectives could be divided into two t ypes- administra- 
tive and out come, t , , t , , division is partially based upon 
the degrees of difficulty in evaluation. 



EVALUATION 



WHAT IS EVALUATION? 



Evalu, O, can be defined as a technique which provides « decisionmaker with information about the merit 
Of plan, the processes being utilized, or, a product that has resulted from activities. The evaluation of n 
prov.de, information about the worthed value of goals, objectives, and .trategic. This proc „ s " fid to 
s .nput evaluate and is accomplished by expert review of plan, and resources. The details of input ,vah, ; , 
tion are not discussed in this work because the focus of this monograph is on evaluating the effectiveness of 
operates Tcch-nques for collecting information on project effectiveness „ hereafLr called ™ „ 
product evaluation Evaluation techniques are, in this model, always related to objectives 

/WW Evaluation occurs when one monitors the daily operations of a project. It is used to assess 
whether act,™ strategies, and treatments are working on n daily basis as planned .The cone 1 Z\ y 

mple but , can be rune consuming, since frequent collection of data is required. Chart 4 ( ' ,,) no d 
an example of how to perform process evaluation. P S ' provl ■ 

There are many kinds of indices one could utilize when collecting data for each objective being subjected 
o process evaluation. The objective in Chart 4, for example, relate! to increasing a'child's abl d 
h mseK. Information about this objective could be collected in a variety of environments the home the 
ch™„, on f.e.d (rl p S . Obviously, this data provides feedback as to how well the child is doing Th d 
could be summarised to report how well the project is doing in meeting this objective for all children and in 
this way, pro V1 de information for another area-product evaluation 

Whereas process evaluation provides information far daily decisionmakers, product evaluation is a 
Q . mary of mformat.on taken over a longer time frame such as one month, six months, or one y r Produc 
ERIC»«'«| ;«vea| S information about what the target population was lib when « program started, and w 
populate 1S l,ke after treatment. Whereas process evaluation focuses on the effects of trcatmenT o, 
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Individuals lor brief periods of time, product cvuU 
nation is used to produce data about changes in 
groups of people over periods as long as one year. 



CHART 4 

SAMPLE OF PROCESS EVALUATION 
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OBJECTIVE 


TASK 


MONITOR 


SAMPLE DATA 


To increase each child's 


Child able to feed self. 


Anecdotal record for one 


Oct. ! - Child can feed 


self-help skills so that by 




chili 


self finger food, 


June 1, he can feed him- 








self without the aid 'of 






Oct, 4 — Child interested 


others. 






in lioldiiig spoon, but not 








able. 








Oct. 10 Child holds 








spoon poorly, drops often. 



Chart 5 (page 1 5) illustrates that information 
gathered for process evaluation (in this case the 
anecdotal record) can be summarized for product 
evaluation results, 



IS 



Chart 6 (page 16) presents an overview of 
potential product evaluation techniques that one 
might use to obtain data on the outcomes of ex- 
tended treatments. Techniques are classified 
according to type of measurement device (stand- 
ardized or non-standardized) and by target group 



(children, parents, or deeision^nakers). Note that 
most of these techniques could be used in process 
evaluation, For example, attendance reports and 
records of contacts can be used to indicate how 
treatment of programs are progressing, A summary 
of this information demonstrates achievement of 
the rulnted objectives. 



CHART 5 

SAMPLE OF PRODUCT EVALUATION 



OUTCOME 
OBJECTIVE 



To increase all children's 
self-help skills so that by 
June 1 they feed selves 
without the- aid of others. 



TASK 



Children able to feed 
themselves unaided. 



MONITOR 



Pre=post observation 
of criterion behav- 



SAMPLE DATA 



lor. 



Anecdotal record. 



On November 1, only 10% 
of the children could eat 
unaided (N=25), By June 
1, 96% of the children 
were eating without the 
aid of others, 
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CHART 6 

TECHNIQUES FOR PRODUCT EVALUATION 



MEASURES 



CHILDREN 



PARENTS 



DECISION MAKERS 



STANDARDIZED 



NON 
STANDARDIZED 



Standardized tests ad- 
ministered to child 

S ta ndardized infornK 
ant=interview scales 

Established observation 
and behavior analysis 
schemes 



S ta rid a r dized "papeN 
and- pencil" measures 
administered to parents 



Established observation 
and behavior analysis 
schemes for parent- 
child interaction 



Attendance at program 

Number of children 
sent on to regular class- 
rooms 

Parent report-checklist^ 
rating scales, letters 

Teacher report, check- 
Hit, rating progress re- 
port scales 

Anecdotal records, 
compiled 

Case studies by elini* 
clan 



Attendance at parent 
meetings 

Record of e on tacts 
with center: 

Phone Calls 

Appointments 

Conversations 

Visits to classroom 

Interview of n on- stand- 
ardized paper and pen- 
cil tests on i 

Attitude 

Knowledge 

Report of chiW- rear- 
ing practices 

Opinion of parent 
child program 



Recording number ofi 
Requests of center 
for consultation 

Referrals of children 
by other agencies 

Observations arid 
visits to school 

Brochure circulation 
Speaking engage- 
ments by staff 

Reports in news- 
paper articles, T.V\ stor- 
ies, magazine pieces, 
articles in journals 

Contacts with other 
age ncies 

Presentations to 
groups 
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MEASURES 


CHILDREN 


PARENTS 


DECISIONMAKERS 


NON 
STANDARDIZED 

(con~t t ) 


Records of criterion be- 
havior 

I n f o r m it n t i n t ervie w 
scales 

Criterion test adr in is- 

tCFDn tn rhild 

l VI b U LU L 1 1 1 1 LI 

Observation and behav- 
ior analysis schemes, 
applied to behavior ob- 
served on video tape 

Letters on testimonials 
from those who know 
dhild 


Teacher report of par- 
ent behavion 

Anecdotal records 

Checklist 

Descriptive narrative 

Observation and analy- 
sis of parent-child inter- 
action 

Letters and comments 
from parents 


Questioning parents on 
how they learned of 
program 

Follow-ups of work- 
shops arid demonstra 
tion activities for feel- 
ings about presentation 

Record of new facili- 
ties* modeled after 
yours 

Noting changes in bud- 
geting that increase 
fundi to handicapped 

Straw polls of agencies 
and city about know- 
ledge of project 
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IS EVALUATION THE SAME FOR ALL 
ADMINISTRATIVE AND OUTCOME 
OBJECTIVES? 

The previous chapter Indicated that ohjec- 
rives could be divided into two fcypcs= administra- 
tive and outcome (see Figure 2). This division is 
partially based upon the degree of difficulty in 
evaluation. The achievement of administrative 
objectives is less difficult to assess since they are 
related to management. On the other hand, eva!ua= 
tion of outcome objectives focuses on changes in 
the behavior of a target group. Thus* measurement 



of a more sophisticated nature is needed, 

Most administrative objectives require v?*ry 
specific actions and results, such as, to select 25 
children for program* hire two early childhood 
specialists, develop a language curriculum. Whether 
or not these objectives arc met can generally be 
icported in a "yes" or 4 *no"" fashion or illustrated 
with the descripdon of some artifact, such as man- 
ual for the teaching of language. The procedures 
necessary for collecting data for this evaluation 
can generally be satisfied by recordkeeping and 
counting (see Chart 7, page 19), 



FIGURE 2 

RELATIONSHIP OF OBJECTIVES TO 
STRATEGIES AND EVALUATION 



Administrative 
Objectives 



Manage me nt 
Strategies 



Administrative 
Evaluation 



Goals 



Treatment 
Strategies 



Behavioral 
Evaluation 



Outcome 
Objectives 
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CHART 7 

EVALUATION OF ADMINISTRATIVE OBJECTIVES 



OBJECTIVE 



To have one parent from each family spend 1/2 
day per week observing teachers at center 



To make 10 other agencies aware of the services 
we offer and to have 5 of them make referrals 



POSSIBLE 
MONITOR 



by 6/1/72 



Attendance 
record 



Records of 
referral 



SAMPLE DATA 



Smith 
Jones 



1 0/8 
x 



Agency 
Social 

Dept. of Ed, 
Mental Health 



WEEKS 
10/15 



I -St Refer™ 
6/1/72 
1/20/72 
2/1/72 



10/22 
x 

X 



Total 

S 
I 



The* evaluation of outcome objectives in 
much more complex since it deals with human 
behavior. Such objectives as to increase the expres- 
sive language of language-impaired children by a 
significant amount by June 1, or decrease by 50% 
the inappropriate behaviors of emotionally dis- 
turbed children by May 1, might require educa- 
tional testing or observational analysis. Because 
these objectives are difficult to assess, the use of 
several informal measures as a supplement to 
sta n d a rdiv ed measurements is recommended, 
Chart 8 (page 20) demonstrates me clustering of 
two measures to reveal information about one 
objective. 



might use for evaluation. Some projects have a 
staff evaluator who is responsible for all data col- 
lection and reporting, A few projects have used 
outside consulting firms whose personnel design 
the data and prepare the report, Finally, others 
have combined the skills of their own personnel 
with those of an outside evaluation consultant or 
group. Any of these sources can produce good 
results and each project will have to decide which 
method best meets its needs, 

As for the kinds of data collection and sug- 
gestions for reporting procedures, Chapter III will 
deal with these subjects In greater detail, 



WHO SHOULD CONDUCT THE EVALU- 
ATION? 

At present, there are several sources a project 



so 



CHART 8 

EXAMPLE OF EVALUATION OF OUTCOME OBJECTIVE 



TARGET 


OBJ ECTIVE 


TECHNIQUE-PRE-POST 


SAMPLE DATA 


E ni o t i u nail y d I h- 
turbed c h i Id run 


To decrease ihiip pro- 
pria tti behaviors of 
all children in the 
project so that they 
are able to function 
emetic n a 1 1 v and 
socially in group 
activities 


0 b sv rvn t k ma I A i \a lysis 
Teacher's aide charts behav- 
iors f i r 3-day period, 1 hour 
per day in group setting in 
October, January, March. 
A minus one score is given for 
each inappropriate behavior, 

Anecdotal records 
Anecdotal records are kept 
on each child for the year. 
The records of the week and 
one weekly chart are given to 
three outside observers for a 
blind comparison ol which is 
behavior more appropriate. 


At the start of the year each 
child was observed in group 
activity so that a baseline 
score for disruptive behavior 
could be obtained. Disruptive 
behavior was defined as be= 
Iiavior which required the at- 
tention and correction of the 
teacher. The initial group 
score was -95. Bv January 
this score was 35 and in 
March, 5, Anecdotal records 
indicate that all but 2 of 25 
children are now considered 
to be functioning well in 
group activities as compared 
with 1 5 to 25 in October, 



, , . data from a network of projects aid the funding agency 
in its advocacy for the projects arid their programs. 
. . . number of levels at which request for data can he made 
cvhicci why demands for conflicting data arc sometimes 

, . , *jec fives are evaluated to provide internal information 
on the results of treatment; however, this information is 
alw used to satisfy the demands of outside audiences for 
data on ihe re nulls of ihe project's work with a target popu- 
lation. 



DATA COI^IjElCTIOIsr 



WHY ARE REQUESTS FOR DATA AND INFORMATION MADE? 

Projects in the First Chance Network need to collect at least two kinds of data -one to meet their own internal 
evaluation and decision-making needs and the other to meet requests by external agencies, such as the Office 
of Education, 

The most obvious reason a funding agency requests data from local projects is that demands are made on 
the agency for information about how monies are being spent and what services are being rendered. Generally, 
the Office of Management and Budget, the federal fiscal "watchdog", is the initiator of demands for data, but 
others that frequently request information might include the Congress, the White House, or the Program 
Planning Branch of the Bureau of Education for the Handicapped. Second, data from a network of projects 
can help the agency decide how to allocate future funds to help programs meet their objectives, and how to 
best use its own personnel. Finally, data from a network of projects aid the funding agency in its advocacy for 
the projects and their programs. When the time comes for a policy decision on budgetary matters, good data 
could greatly influence the allocation of resources to the entire network, 

WHY DO DATA DEMANDS VARY SO MUCH IN TYPE AND REQUIRED ACTIVITIES? 

Data requests from a parent agency to a local project may seem to be extremely inconsistent. For 
example, a project director may be asked if he needs help in establishing program components, and at the same 
Q he may be told to prove he is having success in working with children, The problem is one of having to 
ERJC Je " ata to the funding agency about areas of potential weakness while ac the same time having to 
HaMHMmsErate success in the same areas, The director is in a dilemma about how much "weakness" data to 
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reveal and how to emphasize the ■'strength 1 * data. 
Usually, the two kinds of data arc not solicited by 
the same source but represent questions and 
demands from different sources within t the same 
govern mental structure. 

The variety of offices that can influence 



inquiries for data in the First Chance Network is 
shown in Figure 3. The number of levels at which 
requests for data can be made evinces why de- 
mands for conflicting data are sometimes made. 

Examples of evaluation questions and data in 
which decision-makers at various levels might have 



FIGURE 3 

OFFICES THAT CAN INFLUENCE INQUIRIES FOR DATA 



CONGRESS 




SECRETARY 
HEW 




OFFICE OF MANAGE- 
MENTS BUDGET 



COMMISSIONER 



DEPUTY 
COMMISSIONER 



BUREAU 



DIVISION 



BRANCH 



SECTION 



1ST CHANCE CENTER 



FIRST 
CHANCE 
NETWORK 
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an intei est appear in Chart 9 on page 27, Sample 
procedures for collecting data are also provided, 

WHAT ARE THE MAJOR TYPES OF 
DATA TO COLLECT FOR EVALUA- 
TION? 

There are three basic types of data which a 
project could pass on to its funding agency or 
other sources it wishes to influence— input, prt%- 
and product. 

Input data describes the resources available 
in the operation of a project, like salary of teaelh 
ers, number of personnel, number of students. 
This kind of data is relatively easy to collect and 
analyze, Generally, this data will be requested 
from projects on a yearly basis and will be used for 
surnmative information about the network, not for 
the evaluation of the success of the project, 

Process data provides information on the 
actual daily operation, treatments, and strategies 
of programs, The collection of this kind of infor^ 
mation requires the specification of what is cx- 
pectcd to occur daily and a method of cheeking to 
see if it did, While useful for decision-making with- 
in the project, this type of data in raw form, (daily 
attendance records of parents in classroom) may 
have little meaning for anyone other than those 
closely associated with the project, But surnmative 
information (total number of parents attending 
class for year in class) from process evaluation may 
be useful as product data. 

Product data describes how effective the 
project was in meeting, its operational objectives. 
For example, if one of the objectives of the proj- 
ect was to have 75% of all children feeding selves 
by June 1, 1972, and on June 1 they found that 
80% of them could; then, they can report that the 
^^jeetive has been achieved. Projects developing 
j£is type data will probably find it useful in "sell- 
their programs to the funding agency and 



other interested parties 

While all three types should be collected, 
process and product data, can be especially useful 
in meeting varying requests for data by the fund- 
ing agency, 

HOW MIGHT ONE REPORT INFORMA- 
TION TO B.E.H. OR OTHER INTER- 
ESTED AUDIENCES? 

The answer to this question has been the 
latent subject of the first three chapters. In Chap- 
ters I and II a system which related planning to 
evaluation was discussed, Planning was presented 
as a process of decision-making resulting in the 
organized statements of the goals and objectives of 
a project. One purpose of evaluation is to provide 
information to a decision-maker about processes 
and products. So that ultimately, this information 
can be used again in the planning cycle, 

The procedures described became the bases 
for reporting information to other audiences, Fig- 
ure 4 is a visual display of how this might work. 
Outcome objectives are evaluated to provide in- 
ternal information on the results of treatments; 
however, this information is also used to satisfy 
the demands of outside audiences for data on the 
results of the project's work with a target popular 
tion. At present, this information is required by 
B.E.H, once a year, Administrative objectives are 
evaluated to provide internal and external informa- 
tion on how well the project was able to meet in 
management goals and objectives. In the past, 
has requested this information on a quar- 
terly basis but future plans call for it to be col- 
lected less often. 

The next and final chapter addresses itself to 
some of the current trends and potential problems 
in educational planning and evaluation. Overall, 
this closing portion of the monograph is more gen- 
eral in nature than the first three chapters. 



FIGURE 4 

OVERVIEW OF USE OF GOALS, OBJECTIVES, AND EVALUATION 
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DECISION MAKERS' AND EVALUATION QUESTIONS 
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DECISION MAKER 


QUESTIONS 


DATA NEEDED 


PROCEDURES FOR 
DATA mi I PfHrrnM 




AGENCY HEAD 
POLICY LEVEL (.such 
as the Director of 
&E.H.) 


Which are most impor- 
tant goals? 

Which programs can 
meet the goals most ef- 
fect ively? 

Which of my programs 
can bo defended effec- 
tively? 


Establishment ofpriori= 
ties by political and 
peer consensus 

The relative costs of 
potential program alter- 
natives 

Evidence that programs 
are meeting objectives 


National Advisory 
Groups and Consultants 
meet &i provide consen- 
sus 

Cost effectiveness anal- 
ysis 

Data from programs rel- 
ative to success, Case 
histories and statistics 
needed 




PROGRAM HEAD - 
NETWORK LEVEL 
(Director of 1st Chance 
Model Centers) 


Which of the individual 
sites are of high qual- 
ity? 

Which elements need to 
be changed to strength- 
en program? (Staffing, 
operations, consulta- 
tion) 


Data related to criteria 
of high performance ex- 
pectations 

Data on eon si stent 
strengths and weakness- 
es across program units 


Site visits and panel re- 
views 

Reports from staff vis- 
its, progress reports, re- 
newal applications 






PROGRAM DIREC- 
TOR <- FIELD PRO- 
GRAM (Director of a 
1 st Chance Projeci) 


Are my goals worth- 
while and feasible? 

How can I operate my 
program more effective- 
ly? 

Am I really having in> 
pact on the problem I 
am attacking? 


Local Advisory Groups 
support of goals 

Management analysis of 
cost vs. performance 

Data on change or on 
criterion performance 
by children or ultimate 
target of program 


Continued consultation 
and advice from citi- 
zens and professional 
peers 

Evaluation of effective- 
ness of management ob- 
jectives = pert charts, 
time lines, etc, 

Evaluation plan for im- 
pact and change for 
major program targets 







CHAPTHR IV' 



. . . we believe that the emphasis on planning and evaluation 
comas from a very different source than other educational 
trends and it is quite likely to be around for a long time, 
, . , the unique demands of today and the future require 
thai we forge a system that can relate effectively to contin- 
uous new developments while preserving the best of what 
we now have. Implicit in such a system is the need for 
systematic planning. 



TRENDS 



We might conclude by reflecting on how and why the great emphasis on planning and evaluation h« come 
abouMn American education. Certainly, the "ebb and flow" of ideas and educational fads are not new to the 
American educational scene. Not too long ago, we had grout stress on team teaching and teaching machines as 
.-. way out of our educational wilderness. We now are in the midst of concerns for open education and open 
classrooms with a predictable future decline of interest in that particular approach. It is tempting to think of 
planning and evaluation" »,ju.t another one of these educational lads. Many educational administrators have 
detcrmmed that if they can just "grit their teeth and hang on", the wave of interest \, evaluation will pas, by 
and they will not have had to respond specifically to it. But we believe that the emphasis on planning'., „d 
evaluation comes from n very different source than other educational fads and It is quite likely to be around 
for a long tune. - ' 

III the I960-* we were going to use the educational system to remedy the social imbalance for those 
-citixanii who had not had fair or equal educational opportunities. Programs like Head Start and Title I of the 
Elementary and Secondary Education Act were designed to bring about a balance, and many other educational 
initiative* had similar objectives. But these program! have net reached the high, if not impossible, goals set by 
then enthus.astic supporters, it should W clc.i, by now that an educational system, by itself, willnet eliminate 
poverty. 

For example, the Coleman Report .Col nw.», et 1 966) lias probably had more influence on decision, 
makers than any proceeding piece o: „ search, fids is not because it is more sophisticated or more excellent 
than other studies, because it is not. it.u, i|,e fesu l ts were interpreted to mean that schools do not make that 
much difference ir. the development of the child; more important than education is the social milieu in which 
FRiri ° ptfratCS * A reee,,t P ublici,ti °» h Jcnto (1972) continues that concept and suggests that education 
hm ,1£tlc mi P aet °» thc development of children and that many other factors are of much greater 
importance, 
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LX)UBTS ABOUT EDUCATION'S ABIL- 
ITY TO IMPROVE 

There can no longer be much doubt that In 
America many groups have grown increasingly 
skeptical about the values of public education. The 
sharp increase in school bond rejections* the lack 
of solid political support tor major educational 
funds, and the new cry for accountability arc all 
symptoms of an important estrangement, 

What does the recent wave of popularity for 
such terms as accountability; performance con- 
trading, and educational vouchers really mean? 
Demands tor accountability represent a suspicion 
that the schools are not doing a good job and sup- 
posedly, accountability reports will provide the 
public witli the evidence to support that suspicion. 

The public seems tired of all the input 
descriptions— the number of teachers, the cur- 
riculum programs, or the new plan=and now 
wants some output figures. In other words, what 
happens as a result of all of this input? The inter^ 
est in per for ma nce=eon tract ing represents a strong 
public willingness to let somebody else take a 
crack at the problemi with which the schools seem 
to be unable, or unwilling, to deal* Likewise, there 
is considerable interest in the concept ot educa- 
tional vouchers which gives the consumer some 
control over how he spends his educational dollar 
and represents the feeling that a ^carrot and stick" 
approach is needed to force change, since the 
schools will not reform themselves, 

We can all agree that the American educa- 
tional enterprise, like other large and complex en- 
tities, does not reform itself too easily. With evi- 
dence all around us of our inability to provide 
good education for the disadvantaged, we don't 
seem to be making or even discussing any great 
changes or modifications. 

In special education there is increasing evi- 



fields. For example, the lack of positive gains in 
the special class programs for educable mentally 
retarded, the limited educational output front 
either the oral or manual approach for the deaf, 
and the spontaneous remission of troubled chil- 
dren without treatment, have not caused a major 
reorganization of services, Is this because of 
stupidity, weakness* or self-interest? What is keep- 
ing us from more intelligent programming? 

We have a characteristically human way of 
dealing with such failures— the decapitation of the 
"guilty". We fire superintendents and chancellors! 
we eiert different public officials in a vain attempt 
to purge ourselves of the evil and incompetent: 
and we seek the charismatic leaders For the sake 
of simplicity we hope that evil men are the root of 
the problem* but history tells us that this really 
isn't so. We suggest instead an alternative hypothe- 
sis to explain the slowness of the educational 
establishment to acts We fail to solve educational 
and social problems because we are not organized 
as a society to solve those problems* 

THE EDUCATIONAL NON-SYSTEM 

The "American Educational System" as a 
concept is both a misnomer and an oversimplifica- 
tion. First, we don't have a true system in the 
usual sense of that word, A system is a combina- 
tion of elements functioning in a relationship to 
one another. Instead of this, we have an educa- 
tional tradition that stresses autonomous units and 
self-contained operations rather than interactive 
mutually responsive elements. There is both 
strength and worth in the American educational 
tradition of diversity^ but the unique demands of 
today and the future require that we forge a sys= 
tern that can relate effectively to continuous new 
developments, while preserving the best of what 
wc now have, Imnlicit in such a system is the need 
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for systematic planning. 

One of the advantages of the planning and 
evaluation model presented In this monograph is 
that it allows us to face up to alternative strategies. 
These strategies may be hidden from us as long as 
we deal only with our own single program, A 
graphic example of the complexities of strategy 
selection in program planning is provided by the 
following discussion of the man-power training 
needs for emotionally disturbed children. 

If we use a conservative incidence figure of 
two percent in the area of emotional disturbance, 
we find Jlat there are approximately 1.2 million 
disturbed children (ages 5-19) in need of special 
services in the United States, If our strategy called 
for the provision of full, special education services 
to all children, we could determine how many pro* 
fessionals would be needed by establishing the 
accepted .staff child ratio. Therefore, we would 
need 150,000 .specialists. But let us continue in 
our conservative mode and say that we will settle 
for giving special service to only sixty percent of 
the emotionally disturbed children by 1975, That 
means we will need only 90,000 specialists, instead 
of 150,000, Right now we have only about 
1 1,000, So we need about 79,000 more specialists 
(see Chart 10, page 32), 

At this time, we can begin discussing money, 
Many people think that if the federal government 
would only give more money for training, maybe 
the problem would be taken care of. But Is money 
the problem? If we use estimates that for every 
federal fellow there are 2 specialists graduating 
from master's programs without support and 4 In 
undergraduate programs, we can estimate that a 
total of 500 specialists per year are being grad^ 
uated, Using 1970 as a base, we will need 158 
years to meet th- demand for personnel to provide 

rn^" erVice t0 Sixty P ercent o{ thG emotionally dk= 
fcKJL turbe ^ or by the year 2128 we will be providing 
msmsem special services to sixty percent of those children 



in this area who need them, 

However, these figures don't reflect the 
yearly manpower attrition rates, We can expect to 
lose about eight pereeiu of the work force in any 
year through death, retirement, and pregnancies, 
which is about 900 persons. Since we are only 
graduating 500 a year, we aren't even keeping up 
with the attrition rate, Doubling the program out- 
put at the federal level would only allow us to 
break oven. It is obvious that we can't begin to 
meet the need for professional services in the 
futu re, 

Through this example one can see that edu- 
cation like any other enterprise, needs facts to 
guide its actions, creative ideas to help it improve 
performance, and careful planning to assure that 
needed resources arrive at the right place at the 
right time. Traditionally the political realities of 
resource acquisition have not allowed realistic, 
systematic planning since the approach to system-' 
atic planning contrasts significantly with the way 
programs are supported now, Figure 5 shows two 
curves in the growth and application of resources 
on a particular hypothetical problem. The solid 
line shows the typical expenditure of resources 
curve that comes from systematic planning. The 
first year or two Is usually spent In the analysis of 
the problem area, the gathering of resources, the 
preparation for a major effort, Once this ground 
work is laid, much greater resources during years 
three and four can be applied to the problem area, 
so that by the time the final resources are needed, 
a rather careful and systematic program growth 
curve is produced, 

In stark contrast to that curve in Figure 5 is 
the dotted line which shows a more typical fund^ 
ing pattern that comes through the political 
process, whether at the state or federal lyvel, 
regardless of which political party is in power. The 
standard political philosophy for the "funding of 
new programs is "strike while the iron is hot", or 



CHART 10 

SPECIAL EDUCATION MANPOWER NEEDS-ESTIMATION 



NATIONAL CURRENT MODEL NATIONAL CURRENT MODEL 



Children needing services (ages 509) 2% 1,200,000 1% 600,000 

Children now receiving special services 90,000 120,000 

Trained professionals available 1 1,000 9,000 

Professionals needed to meet 60 percent of need 79,000 14,000 

(B\ \ ratio) (20:1 ratio) 



Taken from Gallagher (1972), 
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try to get all the resources that you can for 
your program while you can in the fear that the 
public or the legislature or both will lose interest 
in your particular area and will move on to other 
issues. Unless you get major resources white you 
have the attention and interest of the public and 
the legislature, you will be unlikely to get them 
Inter on, Therefore, even though wise legislators 
know that it is difficult to spend large sums of 
money wisely at the beginning of a program; they 
will nevertheless ask for large sums on the grounds 
that, while the funds for the program may not be 
increased beyond the initial level* they will not 
likely be decreased, Large grants are asked for so 
that when planning and experience finally catch 
up with the resources, there will be enough re- 
sources thci^ to deal with the problem, This is a 
very wasteful and discouraging approach to the 
problem, because it encourages inadequate per- 
forma n cc and failure, We must find some more 
sensible approach to resource allocation if the 
cycle of enthusiasm, disappointment, and rejection 
does not follow its predictable course in one pro- 
gram after another. 

The Handicapped Children's Early Education 
Program was more fortunate than most in the 
sense that it had a short early growth period and 
the chance to work out many of its problems be- 
fore it became a large and major demonstration 
network. Those who are familiar with the projects 
and the programs involved in the centers can 
probably predict the large number of additional 
problems that would have oceured if enough 
money had been available in the first years to fund 
one hundred centers instead of twenty, 

PROBLEMS WITH PLANNING 

In the final analysis, the great enthusiasm* in 
some quarters, for systematic planning should not 



hide from us a number of weaknesses and prob- 
lems. 

First of all, there is a tendency to display 
goals, objectives, and strategies— the paraphernalia 
of planning— without drawing a tight relationship 
between such statements and actual budget alloca- 
tions, It is generally accepted that stating objec- 
tives and strategics in the absence of budget alloca- 
tions is not a plan but a dream* 

Another common error involving planning is 
the belief that the establishment of a three- or 
five-year plan is a prediction of what will occur. 
We are fiat really attempting to predict the future 
with long-range planning. Instead, planning allows 
us to see what we will need, if you carry your 
objectives through to the end result, !t is no sur- 
prise that the vast majority of five-year plans never 
come into being, They serve their purpose by 
allowing the individual who has done the planning 
to have more insight into the wide variety of re- 
sources needed and the necessary sequence to 
bring these resources to bear on the objectives. 

One other problem is the possibility of set- 
ting the plan decided upon in psychological con- 
crete in the minds of those who decide upon it and 
those expected to carry it out. The ability to 
develop a plan that can change with changing cir- 
cumstances and in the face of new data is perhaps 
the prime challenge of the planners of today, 
There is no reason to believe that it cannot be 
done f but the effort, the commitment, and energy 
that goes with the first plan seems to make it diffi- 
cult to change even when such change is clearly 
called for. 

Finally, we often focus on evaluation by 
looking only at the results with the primary objec= 
fives* Secondary results arc hardly ever studied or 
analyzed. For example, in evaluating an early 
childhood program for handicapped children, the 
most common efforts to evaluate focus on changes 
In the developmental patterns of the children, 



Changes in the parents, or in the neighborhood or 
community are rarely^ il ever, pursued. 

Similarly, a secondary or tertiary effect such 
us the development of leadership people, through 
the stimulation of special programs such as the 
Early Childhood program or Title III, ESEA, rare- 
ly becomes a part of the total evaluation. We run 
the risk of drastically underestimating the program 
effects, because in many cases the second and 
third order effects are more important than the 
primary objectives themselves. Such observers as 
Scriven (1967) have been so impressed by this 
problem that they have suggested that evaluation 
should be done without paying attention to the 
stated objectives of the program at all so as to 
avoid "tunnei" vision, or seeing only those dimen= 
si ons that the original program focused on, 

Needless to say, these lists of problems sug- 
gest that by becoming interested in planning, we 
have exchanged one set of difficulties/or another. 
The most we can hope for is that these difficulties 
will be not as severe as we faced whenever we 
handled our programs haphazardly. 
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GLOSSARY 



Activities - 



Alum**. S,™^/„ -Tl„ pw.nd.l ,„„„, by whlcl , „,„ goa|s „ f „ ^ c<ju|J ^ ^ 



Constraints 
Evaluation = 
liven t — 
Feedback — 
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frnTlirVlT en .r°" mC ^ Wh,C J? llmit th ° SC °P° «» d of objectives (lack of 

funds, limited availability of specific typo teacher) 

Delineating, obtaining and providing useful information for judging d.-cision alter™- 

iS'iftf' 5 ' ' CC0,nplil,,mCnl M " at a partly 

Information from evaluation which has implications for future planning 

A general statement revealing assumptions made about expected outcomes of in 
organs program. Few goals a- , needed and they should identify a p ogram .rea ' 
targets, purpose, expected results and expected completion date % ' 



ementation ~ initial operation of strategy 



40 



Inputs— All elements used in the implementation, maintenance, and development of the 

program 

Needs — Areas void of* or weak in services, production, or development 

Objectives = Statements written in behavioral terms which describe the results of planned activities 

and events 

Administrative Objectives are necessary for the establishment, organization and main- 
tenance of the human and technological systems in each project, A project just begin= 
ning would probably have many of these objectives (to hire staff, develop curriculum, 
implement parent program). 

Outcome Objectives reveal the specific behaviors with which the program is to deal 
and indicate expected results. Like administrative objectives, these reflect the choice 
of strategies the project has made. Whereas administrative objectives are related to the 
collection, allocation and use of resources, outcome objectives reflect what target 
behavior is to be affected by those resources and in what direction. 

Output - Results of activities and events reported in terms of productivity, development, and 

sentiments / 



Rationale - Statements of philosophical, psychological, theoretical, research and pragmatic reasons 

for the implementation of a program 

Resources — The human, technological and organizational materials available for use 

Selection Criteria — Bases for the selection of a particular strategy 

Strategy - A "plan of attack", a method or procedure ior determining what set(s) of activities- 

will be utilized 



Targets - 



Those individuals, organizations or groups which a project objective purports to effect 
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APHBNDIX B 



EXAMPLES 

OF GOALS, OBJECTIVES, STRATEGIES, AND EVALUATION 



GOAL 



The project will active- 
ly attempt to eneourAge 
and create replication 
of its philosophy, pro- 
grams, and components 
of programs. 



OBJECTIVES 



To increase the com- 
munity support of pre= 
school education by 
disseminating Informa- 
tion on the philosophyj 
of the project, success 
with young children, 
and its potential worth 
to the total commun- 
ity. 



STRATEGY 



Weekly 14-hour TV, 
show featuring support 
ivc parents and other 
local persons, 

B i- m o n thly news re= 
leases which emphasize 
human interest success 
stories, 

Workshops given to 
local businessmen and 
political leaders empha- 
sizing the cost-effective* 
ness of* the preschool 
approach, 



EXAMPLE 1 



EVALUATION 



A questionnaire on atti- 
tudes toward, and sup- 
port for preschool edu- 
cation to be given to a 
random sample of 
150^200 local citizen* 
at the beginning of the 
project and yearly 
thereafter, 
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EXAMPLE 2 



ERIC 



GOAL 


OUTCOME 
OBJECTIVES 


TREATMENT 
STRATEGY 


EVALUATION 


The model project 

r j - - - 

through its Parent Pro- 
gram Information Ex- 
change, will increase 
parents* knowledge 
about and acceptance 
of, their child, 


By the end of the first 
year of the Parent Pro- 
gram, parents enrolled 
in the program will in- 
crease their knowledge 
of child growtli and 
development by 30%. 

By the end of the 
second year of the proj- 
ect, parents' long range 
expectations for their 
ha ndleapped children 
will shift in a more real= 
istic direction. 


Group meet! ngs be= 
tween staff and parenti 
in which the con tin u= 
ous growth and devel- 
opment of the child is 
discussed and the pro- 
gram is explained by 
the staff 

Periodic meetings 
planned and led by par- 
ent members in which 
they discuss their chil- 
dren, present and 
future 


Increase in knowledge 
of growth and develop- 
ment is measured by a 
eriteria-refereneed pre 
and post tests. 

Long=range expecta- 
tions are assessed by 
pre and post written ex- 
pectations by parents 
and judged indepen- 
dently by two project 
staff members having 
daily interaction with 
the children, 
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GOAL 



Project's Parent Pro^ 
gram will reduce anxie- 
ties caused by fear or 
guilt feelings due to the 
pressure of a handi- 
capped offspring in the 
family. 



OUTCOME 
OBJECTIVES 



To reduce, by the end 
of the second year of 
the project, anxiety by 
a significant amount in 
90% of the parents, 



TREATMENT 
STRATEGY 



Parent group discussion 
In which parents discuss 
their efforts to help 
their child on the prob- 
lems they have encoun- 
tered in such effort 

Social worker assigned 
and available to each 
parent for two hours a 
week for individual 
counseling. . 



EVALUATION 



Records will be kept 
listing parents who par- 
ticipate and their time 
of involvement. 

Anxiety levels will be 
measured by a scale 
(the IPAT 8-ParuIIe] 
Form Anxiety Battery) 
as the parents enter the 
program and at the end 
or the second year of 
participation, 
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ADDITIONAL MATERIAL 

ON GOALS AND OBJECTIVES 



EXAMPLE 1 

DISTINCTION BETWEEN GOALS AND OBJECTIVES 





GOALS 


objectives 




Show general intent and direction 


Some specific intentions with measurable in- 
dices and time limits 




The project will improve language development 
in young handicapped children 

The parent program will increase parental in- 
volvement with children 


To improve, beyond normal expectations, the 
, receptive vocabulary and complexity of expres- 
sion in retarded children in our center by June 
1, 1973 

To increase over baseline performance parental 
verbal interaction (non-hostile) with child by 
June 1, 1973 


ERIC 
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EXAMPLE 2 

DISTINCTION BETWEEN 
ADMINISTRATIVE OBJ ECTIVES 
AND OUTCOME OBJ ECTIVES 



ADMINISTRATIVE OBJECTIVES 



Useful for program management and progress 
reports 



OUTCOME OBJECTIVES 



Represent final statements of expected benefits 
from the project. Final report materials 



Counseling group fur parents that meets once a 
month during 1 973 

To hire three qualified speech teachers arid two 
aides to deliver service to the children by May 
1, 1973 



To increase parents' personal, non-hostile inter- 
actions with their children by 25% over baseline 
by June 1, 1973 

To improve by a statistically significant amount 
the expressive language skills of children during 
this school year 



EXAMPLE 3 



DIMENSIONAL OVERVIEW OF GOALS 
AND OBJECTIVES 
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GOALS 


ADMINISTRATIVE 
OBJECTIVES 


OUTCOME 
OBJECTIVES 


TIME SPAN 


Expansive (Years) 


Brief (Months) 


Moderate (] year) 


CONTENT 


Revca Is assumptions 
made about the future 
effect of project's pro- 
gram treatment 


Reveals events which 
must occur before pro- 
gram outcome can be 
achieved 


Reveals behavior or 
attitude changes which 
are a result of program 
treatment 


EVALUATION 


Expert review 


Frequency count 

"Yes/no" 

Log 

Check list 
Criterion reierence 


Psychological tests 
Center-made tetts 

Criterion= referenced 
tests 

Frequency count 
Testimony 


NUMBER 


Few (2-3) 


Many (10=20) 


Several (4=6) 
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EXAMPLE 4 

SAMPLE STRUCTURE FOR A GOAL 



GOAL (Model) 

The _ Project through its 

(Title) 

- . . ^ Program for . 



(Descriptors) - (Whom) 



(Action Verb) (What) (For What) 



(General Purpose) 



GOAL (Example) 

The Newton Project through its Developmental Training Program for exceptional f q iter e hij d rgn will 
(Title) - (Descriptors) (Whom) 

increase the dev elopmental level of project children so that t he children remain in trie fo ster home, 
(Act, Vb.) (Whan) (For What General Purpose) 



VERBS USED WITH GOALS 
• provide increase 
promote decrease 



enable 
change 



offer 
train 
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EXAMPLE 5 

SAMPLE STRUCTURE FOR ADMINISTRATIVE OBJECTIVE 



ADMINISTRATIVE OBJECTIVE (Model) 




To _ 




(IitipIeinL-ntating Verb) (What) 


(For 


What Outcome) 


(By When) 


ADM! NISTR ATI V E OBJ ECTI VE (Example) 




To develop a continium of^rogrami to meat the learning needs 
(ImpL vb.) (What) 


of children served by the project [Q-(\ 
(For What 


y^.pi ?i e j by June 1, 1972. 
Outcome) (When) 





VERBS USED WITH ADMINISTRATIVE OBJECTIVES 

coordinate interview create p repare 

or S anizc construct initiate devise 

investigate identify refine develop 

establish individualke plan 

obtain compose implement 



locate 



FRir 



EXAMPLE 6 

SAMPLE STRUCTURE FOR OUTCOME OBJECTIVE 



OUTCOME OBJECTIVE (Mode!) 






To 






(Action Verb) (What Behav 


ior) (Of Whom) 


(With What Specific Results) 


(By When, 




OUTCOME OBJECTIVES (Example) 






To increase the expressive language of each child so that est 


sressive language is within four 


months of his 


(Act, vb.) (What Behavior) (Of Whom) 


(With What General ai 


id/or Specific 


receptive language level by the end of the year, 
Results) (By When) 






VERBS USED WITH OUTCOME OBJECTIVES 

involve stimulate acquaint 
inform change improve 
access increase prevent 


reduce 

liberalize 

decrease 


. • • : • 1, 





